Dual-space high-frequency learning for transformer-based MRI super-resolution.

BACKGROUND AND OBJECTIVE: Magnetic resonance imaging (MRI) can provide rich and detailed high-contrast information of soft tissues, while the scanning of MRI is time-consuming. To accelerate MR imaging, a variety of Transformer-based single image super-resolution methods are proposed in recent years, achieving promising results thanks to their superior capability of capturing long-range dependencies. Nevertheless, most existing works prioritize the design of transformer attention blocks to capture global information. The local high-frequency details, which are pivotal to faithful MRI restoration, are unfortunately neglected. METHODS: In this work, we propose a high-frequency enhanced learning scheme to effectively improve the awareness of high frequency information in current Transformer-based MRI single image super-resolution methods. Specifically, we present two entirely plug-and-play modules designed to equip Transformer-based networks with the ability to recover high-frequency details from dual spaces: 1) in the feature space, we design a high-frequency block (Hi-Fe block) paralleled with Transformer-based attention layers to extract rich high-frequency features; while 2) in the image intensity space, we tailor a high-frequency amplification module (HFA) to further refine the high-frequency details. By fully exploiting the merits of the two modules, our framework can recover abundant and diverse high-frequency information, rendering faithful MRI super-resolved results with fine details. RESULTS: We integrated our modules with six Transformer-based models and conducted experiments across three datasets. The results indicate that our plug-and-play modules can enhance the super-resolution performance of all foundational models to varying degrees, surpassing the capabilities of existing state-of-the-art single image super-resolution networks. CONCLUSION: Comprehensive comparison of super-resolution images and high-frequency maps from various methods, clearly demonstrating that our module possesses the capability to restore high-frequency information, showing huge potential in clinical practice for accelerated MRI reconstruction.

Easy2Hard: Learning to Solve the Intractables From a Synthetic Dataset for Structure-Preserving Image Smoothing.

Image smoothing is a prerequisite for many computer vision and graphics applications. In this article, we raise an intriguing question whether a dataset that semantically describes meaningful structures and unimportant details can facilitate a deep learning model to smooth complex natural images. To answer it, we generate ground-truth labels from easy samples by candidate generation and a screening test and synthesize hard samples in structure-preserving smoothing by blending intricate and multifarious details with the labels. To take full advantage of this dataset, we present a joint edge detection and structure-preserving image smoothing neural network (JESS-Net). Moreover, we propose the distinctive total variation loss as prior knowledge to narrow the gap between synthetic and real data. Experiments on different datasets and real images show clear improvements of our method over the state of the arts in terms of both the image cleanness and structure-preserving ability. Code and dataset are available at https://github.com/YidFeng/Easy2Hard.

Selective Guidance Normal Filter for Geometric Texture Removal.

There is typically a trade-off between removing the detailed appearance (i.e., geometric textures) and preserving the intrinsic properties (i.e., geometric structures) of 3D surfaces. The conventional use of mesh vertex/facet-centered patches in many filters leads to side-effects including remnant textures, improperly filtered structures, and distorted shapes. We propose a selective guidance normal filter (SGNF) which adapts the Relative Total Variation (RTV) to a maximal/minimal scheme (mmRTV). The mmRTV measures the geometric flatness of surface patches, which helps in finding adaptive patches whose boundaries are aligned with the facet being processed. The adaptive patches provide selective guidance normals, which are subsequently used for normal filtering. The filtering smooths out the geometric textures by using guidance normals estimated from patches with maximal RTV (the least flatness), and preserves the geometric structures by using normals estimated from patches with minimal RTV (the most flatness). This simple yet effective modification of the RTV makes our SGNF specialized rather than trade off between texture removal and structure preservation, which is distinct from existing mesh filters. Experiments show that our approach is visually and numerically comparable to the state-of-the-art mesh filters, in most cases. In addition, the mmRTV is generally applicable to bas-relief modeling and image texture removal.